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Determining  the  accuracy  and  limits  of  predictability 
of  numerical  weather  prediction  for  regional  tactical  ihilitary  operations 


EXECUTIVE  SUMMARY  OF  FINAL  REPORT 

The  U.S.  Department  of  Defense  aims  to  develop  capabilities  for  the  numerical  prediction  of 
small-scale  weather  as  part  of  its  Theatre  Battle  Management  project.  This  is  accomplished  at 
research  centers  run  by  each  branch  of  the  military.  Perhaps  the  most  significant  unknown  at  this 
point  is  the  specific  accuracy  of  the  short-term,  high-resolution  models  and  the  extent  to  which  the 
these  forecasts  can  provide  the  guidance  needed  for  tactical  military  operations.  This  report  provides 
a  detailed  description  of  the  research  carried  out  under  an  Department  of  Defense  Experimental 
Program  to  Stimulate  Competitive  Research  (EPSCoR)  grant  fimded  by  the  Air  Force  Office  of 
Scientific  Research.  The  overall  objectives  of  this  research  project  were  to  perform  basic  research 
into  development  of  statistical  methods  for  model  validation,  produce  research  quality  data  sets  that 
could  be  used  for  operational  verification,  examine  issues  in  uncertainty  characterization,  and  to 
identify  and  examine  the  accuracies  and  uncertainties  in  the  operational  models  currently  being  run 
by  the  Air  Force  Global  Weather  Center  and  the  Army  Research  Laboratory  using  the  Oklahoma 
Mesonet  and  other  data  sets.  These  research  efforts  have  been  carried  out  through  the 
interdisciplinary  Environmental  Verification  and  Analysis  Center  (EVAC)  at  the  University  of 
Oklahoma  which  was  established  to  develop  and  apply  geostatistical  methods  to  utilize  point 
measurements  of  geophysical  data  to  verify  model  and  remote  sensing  output.  Below  is  a  short  list 
of  initial  objectives,  a  summary  of  the  key  findings  and  results,  and  list  of  the  research  publications 
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d  presentations  produced  as  a  result  of  this  grant, 
nitial  Research  Objectives 


Develop  statistical  methods  to  quantify  uncertainties  in  surface  validation  fields,  and 
investigate  ways  to  develop  improved  imcertainty  characterization  and  model 
validation  techniques. 

Produce  validation  data  products  at  scale  compai  able  with  model  output  with  known 

error  characteristics. 

Determine  the  errors  associated  with  spatial  averaging  of  surface  and  upper-level 
meteorological  variables. 

Provide  validation  information  for  two  DoD  modeling  groups  (the  Artillery 
Meteorology  Branch  of  the  Army,  White  Sands,  NM  and  the  Air  Force  Global 
Weather  Center,  Ofifutt  AFB)  via  examination  of  their  operation  models  over  the 
Oklahoma  Mesonetwork. 

Evaluate  the  operational  models  in  areas  of  complex  and  mountainous  terrain  (as  a 
simulation  for  areas  of  strategic  military  interest). 


General  Summary  of  Results 

The  following  list  provides  a  brief  overview  of  the  key  findings  of  research  undertaken  for 
this  project.  For  a  more  detailed  description  of  the  methods  and  results,  including  accompanying 
figures,  please  refer  to  the  main  body  of  the  report. 


Brief  list  of  accomplishments  and  achievements: 

Publication  of  seven  referred  journal  papers  related  to  issues  of  statistical  model 
development,  uncertainty  characterization,  and  model  validation 

Presented  several  invited  and  conference  presentation  on  these  themes  (see  list  below 
for  a  more  detailed  description). 

As  stated  in  the  abstract  of  our  proposal:  “The  first  goal  is  to  produce  the  validation 
data  products  at  scales  comparable  with  model  output. ...  This  information  will  then 
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be  used  for  a  validation  exercise  of  the  models  currently  being  used  in  an  operational 
mode.”  These  goals  have  been  accomplished,,  as  we  have  used  the  Mesonet  (and 
other  available  data)  to  identify  the  errors  associated  with  spatial  averaging.  Thus,  we 
have  now  produced  a  valuable  dataset  with  gridded  areal-averaged  values^»fstandard 
surface  meteorological  variables  (e.g.,  temperature,  precipitation,  etc.)  with  known 
error  characteristics.  We  have  also  archived  a  variety  of  atmospheric  sounding  and 
profiler  data,  which  we  are  using  for  the  validation  of  the  Army  Research  Lab’s 
Battlescale  Forecast  Model  (BFM).  This  has  been  done  both  at  the  scale  of  the  BFM 
and  the  Air  Force  Global  Weather  Center’s  MM90  models. 

Completed  a  series  of  operational  tests.  Results  are  quite  promising  in  their  ability  to 
further  improve  weather  prediction  at  increasingly  smaller  temporal  and  spatial  scales 
(see  figures  described  in  main  report  below  as  well  as 
htto  ://www.evac.ou.edu/impbac  for  details) 

In  year  two,  we  sponsored  a  several-months  long  visit  from  a  visiting  scientist  from 
Australia,  Dr.  Bill  Lyons,  who  is  an  expert  in  statistical  methodology  and 
visualization  of  model  and  error  analysis  via  use  of  Geographic  Information  Science 
(GIS)  technology.  Dr.  Lyons  allowed  us  to  develop  a  visualization  and  GIS 
applications  component  to  our  model  validation  research. 

Developed  a  method  to  analyze  the  impact  of  model  errors  on  the  displacement  of 
artillery  shells  (developed  in  collaboration  with  the  Army  Artillery  Meteorology 
Branch.)  The  results  show  the  types  of  weather  conditions  that  provide  problems  for 
the  BFM,  and  have  been  useful  in  examining  the  effectiveness  of  the  model  for  use  as 
a  artillery-directing  tool. 

Performed  a  series  of  model  simulations  and  intercomparisons  between  the  BFM  and 
the  MM-90  (run  with  the  same  configuration  as  the  Air  Force  operational  model). 
Results  illustrate  the  relative  strengths  and  weaknesses  of  the  two  models  for  the  time 
periods  and  spatial  domains  selected.  The  information  produced  from  these 
evaluation  periods  will  assist  the  modeling  groups  to  develop  improvements  to  their 
operational  models.  (See  main  text  below  for  a  full  description  of  the  model 
intercomparison  efforts). 


Future  Directions 

Our  plans  for  the  future  include  ongoing  collaboration  with  the  Army  Research  Laboratory  at 
White  Sands,  NM.  Our  intent  is  to  leverage  our  existing  research  and  EVAC  resources,  and  to 
further  strengthen  and  expand  the  scientific  links  between  the  Army  Research  Laboratory  and  EVAC 
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We  are  currently  working  on  the  development  and  testing  of  new  and  innovative  hypotheses  related 
to  assessing  the  spatial  variation  of  uncertainty  in  the  prediction  of  mesoscale  weather.  In  addition, 
the  model  intercomparisons  that  we  have  planned,  using  surface  error  analysis  and  producing 
statistically  significant  and  meaningful  interpretations,  will  assist  the  Army  in  choosing  their  weather 
forecast  model  of  the  future.  Over  the  past  three  years,  we  have  developed  a  strong  and  close  link 
with  the  Army,  and  we  plan  to  use  this  solid  foundation  for  the  future  in  order  to  pursue  research  into 
issues  of  importance  for  operational  military  forecasts. 
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Results  from  Research  Associated  with  Project 


Publications  '  . 

Published  in  Peer  Reviewed  Journals  and  Books 

Greene,  J.S.,  W.E.  Cook,  D.  Knapp,  and  P.  Haines,  An  Examination  of  the  Uncertainty  in 
Interpolatf'd  Winds  and  their  Effect  on  the  Validation  and  Intercomparison  of  Forecast 
Models,  Journal  of  Atmospheric  and  Oceanic  Technology,  in  press  (to  be  published  March, 
2002) 

Greene,  J.S.,  M.D.  Klatt,  W.E.  Cook,  and  H.J.  Johnson,  1999:  Weather  Modification  on  the  Great 
Plains  (pp.  37-50),  in  The  Ogallala  Aquifer,  Steps  to  Sustainability,  Lori  Triplett,  ed..  Great 
Plains  Foundation,  Overland  Park,  KS. 

Greene,  J.S.,  and  M.L.  Morrissey,  1998:  Evaluation  and  Validation  of  Simulated  and  Observed 
Climate  Data,  in  Climate  Prediction  for  Agricultural  and  Resource  Management,  L.  Leslie 
and  R.  Munro,  eds.,  Australian  Academy  of  Sciences,  Canberra. 

Greene,  J.S.,  and  M.L.  Morrissey,  2000:  Validation  and  Uncertainty  Analysis  of  Satellite  Rainfall 
Algorithms,  The  Professional  Geographer,  52,  247-257. 

Morrissey,  M.L.,  and  J.  S.  Greene,  1998:  Using  the  Oklahoma  Mesonet  to  Develop  and  Test  a 
Sampling  Error  Statistic  for  Meteorological  Time  Series,  Journal  of  Geophysical  Research, 
Vol.  103,  D8:  8979-8984. 

Morrissey,  M.L.,  and  J.S.  Greene,  1998:  Uncertainty  Analysis  of  Satellite  Rainfall  Algorifiims  over 
the  Tropical  Pacific,  Journal  of  Geophysical  Research,  103:  19569-19576. 

Morrissey,  M.L.,  and  J.  S.  Greene,  1998:  Using  the  Oklahoma  Mesonet  to  Develop  and  Test  a 
Sampling  Error  Statistic  for  Meteorological  Time  Series,  Journal  of  Geophysical  Research, 
103,  D8:  8979-8984. 

Morrissey,  M.L.,  and  J.S.  Greene:  Validation  of  Remotely  Sensed  Estimates  of  Geophysical 
Variables  using  Surface  Observations,  Book  in  progress  to  be  published  by  Springer- Verlag 


Published  in  Unreviewed  Publications  (proceedings,  tech,  reports,  etc.) 

“Evaluation  of  the  Battle  Scale  Forecast  Model  over  Oklahoma  with  an  Emphasis  on 
Artillery  Accmacy”,  Greene,  et  al..  Proceedings  of  the  8th  conference  on  Aviation,  Range 
and  Aerospace  Meteorology,  January,  1999. 
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“An  Examination  of  Battlescale  Forecast  Model  Initializations  over  Complex  Terrain,” 
Greene,  et  al..  Proceedings  of  the  9th  conference  on  Aviation,  Range  and  Aerospace 
Meteoro/dgy,  January,  2000.  -  -  - 

Accepted/Submitted  for  Publication 
N/A 

Invention  Disclosures  and  Patents  Granted 
N/A 

Invited  Lectures,  Presentations,  Talks,  etc. 

Invited  Lectures: 

“Statistical  approaches  to  determining  the  accuracy  and  limits  of  predictability  of  mesoscale 
meteorological  models”.  Army  Research  Laboratory,  White  Sands,  NM,  February,  1998. 

“Evaluation  and  Validation  of  Numerical  Weather  Prediction”,  AMS  local  chapter  meeting. 
Las  Cruces,  NM,  Oct.,  1999. 

“Evaluation  and  Validation  of  Simulated  and  Observed  Climate  Data”,  Center  for  the 
Analysis  and  Prediction  of  Storms,  University  of  Oklahoma,  December,  1998. 

Conference  Presentations: 

“Verification  of  a  PC-Based  Mesoscale  Model”,  presented  at  the  1998  National  Weather 
Association  Aimual  Meeting,  Oct.  1998 

“Evaluation  of  the  Battle  Scale  Forecast  Model  over  Oklahoma  with  an  Emphasis  on 
Artillery  Accuracy”,  presented  at  the  Annual  Meeting  of  the  American  Meteorological 
Society,  January,  1999 

“A  Comparison  of  Two  Methods  for  Estimating  Verification  Ballistic  Conditions”,  presented 
at  the  1999  National  Weather  Association  Aimual  Meeting,  Oct.  1999. 

“An  Examination  of  Battlescale  Forecast  Model  Surface  Initialization  over  Mountainous 
Terrain”,  presented  at  the  Annual  Meeting  of  the  American  Meteorological  Society,  January, 
2000. 

“Evaluation  of  mesoscale  weather  prediction  for  regional  tactical  military  operations  with  an 
emphasis  on  artillery  accuracy  assessment”,  presented  at  the  Annual  Meeting  of  the 
Association  of  American  Geographers,  March,  2001. 

Extended  Scientific  Visits  From  and  To  Other  Laboratories: 
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Visit  to  Air  Force  Global  Weather  Center,  January,  1998; 

Visits  to  Army  Research  Laboratory,  White  Sands,  NM,  February,  1998  October  1998;  April 
1999;  October  1999.  - 
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